Component hostcall optimizations - #13876
Conversation
Splitting current_thread into an #[inline] fast path plus #[cold] outlined deferred-promotion slow path reduces call overhead by about 20% (measured in a Linux VM on an M5 Max MBP): sync calls go from about 83ns to about 65ns, immediately ready async calls from 142ns to 117ns.
Precompute `TypeFunc::contains_borrow` at compile time and use it in the hostcall entrypoint to skip `CallContext` scope push/pop and `validate_scope_exit` when lending is statically impossible. Relative to the previous commit, this reduces call overhead by about 29% for sync calls, and about 9% for async calls (both measured in a Linux VM on an M5 Max MBP): sync calls go from about 65ns to about 46ns, immediately ready async calls from 117ns to 106ns. I'm not entirely sure why the win is so much less for async calls, but there are more wins in the next commits.
Such calls now run as the caller's guest thread without creating a pending future at all: `poll_and_block` polls the host future once before materializing anything and returns ready results directly. `host_task_reenter_caller` becomes a no-op when no task was created. Async-lowered calls keep the eager task for subtask status/cancellation), and borrow-ful calls keep it as scope identity. Relative to the previous commit, this reduces call overhead by almost another 60% (46ns -> 19ns) for sync calls, with async calls, expectedly, stay unchanged.
Specifically, if no borrows need to be tracked, no scope needs to be resolved. Plus, inline a bunch of single-instantiation generics. Relative to the previous commit, this reduces call overhead by another 31% for sync calls (19ns to 13ns), with async calls unchanged (both measured in a Linux VM on an M5 Max MBP).
A ready async-lowered call returns Status::Returned with no waitable, so the guest cannot observe the task's absence. first_poll now polls the raw future before creating anything, only pending futures get in `JoinHandle::run` and materialize a task. This only works for borrow-free calls, because for borrows the task is still needed for scope tracking. Relative to the previous commit, sync calls stay unchanged, but async calls are improved by about 45%: 109ns to 60ns. (measured in a Linux VM on an M5 Max MBP)
Polls the result future once in-place and skips creating a `HostTask`, etc if it's ready immediately. Reduces costs of immediately-ready async calls by another 12% (60.5ns to 53ns)
github-actions
Bot
added
the
wasmtime:api
tschneidereit
added
performance
wasm-proposal:component-model
77c8bce to
8da58dc
Compare
|
Thanks! Could this be split up to land each commit independently? Some of these I'm more sure about than others and might take more time to review |
|
Happy to do that, yes! I might in that case also restructure some of them a little bit to bring in some borrows-related fixes which change some of the new code again. |
| mut store: StoreContextMut<'_, T>, | ||
| future: impl Future<Output = Result<R>> + Send + 'static, | ||
| lower: impl FnOnce(StoreContextMut<T>, Option<R>) -> Result<()> + Send + 'static, | ||
| ) -> Result<Option<u32>> { |
There was a problem hiding this comment.
[kinda unrelated to runtime optimizations]
first_poll currently causes a lot of llvm-lines bloat and compile-time overhead.
Do you think it could be reduced by changing generic R to Box<dyn Any + Send + 'static>?
Pin<Box<dyn Future<Output = Box<dyn Any + Send + 'static>> + Send + 'static>>
There was a problem hiding this comment.
Do you think it could be reduced by changing generic
RtoBox<dyn Any + Send + 'static>?
Who knows, maybe? But we'd most certainly have very very compelling evidence that that change doesn't negatively impact runtime performance. Which absent that evidence I would think to be somewhat likely, since it'd introduce a box and a virtual call at a very hot callsite.
This is important to keep in mind for your entire work around reducing the amount of LLVM IR generated: we'd be unlikely to give up even a tiny bit of runtime performance for all but the most massive compile time gains, if at all.
3 participants
This set of commits applies a bunch of independent optimizations to calling host imports from components. In combination, they reduce overhead of sync calls by about 84%, from 83ns to 13ns, and that of immediately-ready async host imports by about 62%, from 142ns to 53ns. All numbers measured in a Linux VM on an M5 Max MBP, but I don't think any of this is particularly architecture-specific.
A few of notes on the results:
All commits can be reviewed independently and come with improvement numbers relative to the previous commit.